Chair of Robotics, Artificial Intelligence and Embedded Systems T echnical University of Munich (TUM) Munich, Germany {yi1228.zhang, erik-leo.hass, Abstract --Autonomous driving systems face significant challenges in achieving human-like adaptability, robustness, and interpretability in complex, open-world environments. These challenges stem from fragmented architectures, limited generalization to novel scenarios, and insufficient semantic extraction from perception. T o address these limitations, we propose a unified Perception-Language-Action (PLA) framework that integrates multi-sensor fusion (cameras, LiDAR, radar) with a large language model (LLM)-augmented Vision-Language-Action (VLA) architecture, specifically a GPT -4.1-powered reasoning core. This framework unifies low-level sensory processing with high-level contextual reasoning, tightly coupling perception with natural language-based semantic understanding and decision-making to enable context-aware, explainable, and safety-bounded autonomous driving.

To build a smarter and safer city, a secure, efficient, and sustainable transportation system is a key requirement. The autonomous driving system (ADS) plays an important role in the development of smart transportation and is considered one of the major challenges facing the automotive sector in recent decades. A car equipped with an autonomous driving system (ADS) comes with various cutting-edge functionalities such as adaptive cruise control, collision alerts, automated parking, and more. A primary area of research within ADAS involves identifying road obstacles in construction zones regardless of the driving environment. This paper presents an innovative and highly accurate road obstacle detection model utilizing computer vision technology that can be activated in construction zones and functions under diverse drift conditions, ultimately contributing to build a safer road transportation system. The model developed with the YOLO framework achieved a mean average precision exceeding 94\% and demonstrated an inference time of 1.6 milliseconds on the validation dataset, underscoring the robustness of the methodology applied to mitigate hazards and risks for autonomous vehicles.

GM's autonomous vehicle Cruise division is already going through a rough patch, with the California Department of Motor Vehicles (DMV) recently suspending its driverless permits over safety issues. Now, several new reports have highlighted other issues with the company, including problems with its autonomous vehicles (AVs) recognizing children and the frequency with which human operators must remotely take control. The company also just announced that it's temporarily suspending production of its fully autonomous Origin transport. The most concerning issue is that Cruise reportedly kept its vehicles on the streets even though it knew they had problems recognizing children, The Intercept reported. According to internal, previously unreported safety assessment materials, Cruises autonomous vehicles may have been unable to effectively detect children in order to take extra precautions.

Over the last several years use cases for robotics based solutions have diversified from factory floors to domestic applications. In parallel, Deep Learning approaches are replacing traditional techniques in Computer Vision, Natural Language Processing, Speech processing etc. and are delivering robust results. Our goal is to survey a number of research internship projects in the broad area of "Deep Learning as applied to Robotics" and present a concise view for the benefit of aspiring student interns. In this paper, we survey the research work done by Robotic Institute Summer Scholars (RISS), CMU. We particularly focus on papers that use deep learning to solve core robotic problems and also robotic solutions. We trust this would be useful particularly for internship aspirants for the Robotics Institute, CMU.

AI that can follow a person seems like a simple enough task. It's certainly a simple thing to ask a human to do, but what if people or objects get in the way of the robot following behind a person? How do you navigate an environment that's in a constant state of change? About a year ago at a robotics conference TechCrunch held at UC Berkeley, AI startup founders explored solutions for common problems encountered when trying to automate construction projects. Tessa Lau, CEO of Dusty Robotics, called attention to the challenge of moving machines in an unstructured environment filled with people.

Almost all robocars use maps to drive. Not the basic maps you find in your phone navigation app, but more detailed maps that help them understand where they are on the road, and where they should go. These maps will include full details of all lane geometries, positions and meaning of all road signs and traffic signals, and also details like the texture of the road or the 3-D shape of objects around it. They may also include potholes, parking spaces and more. The maps perform two functions. By holding a representation of the road texture or surrounding 3D objects, they let the car figure out exactly where it is on the map without much use of GPS. A car scans the world around it, and looks in the maps to find a location that matches that scan. GPS and other tools help it not have to search the whole world, making this quick and easy.

If you're a self-driving car, though, it can be devastating. Work zones flummox the future rulers of our roads because they override or obliterate the sturdy markers by which the vehicles are taught to navigate. With no warning, they enter a world where cones trump double yellow lines, bollards replace curbs, and construction worker hand signals outweigh traffic lights. That's why self-driving pioneers like Google and Delphi cite construction as a common reason their human engineers take control of the wheel while testing: The cues designed for human drivers can stump advanced computer systems. This gets to the central challenge of autonomous driving: How do you teach machines to deal with the chaotic, grubby humanity of our roads, where the rules bend so easily?

Mechanical masterminds have spawned the Bat Bot, a soaring, sweeping and diving robot that may eventually fly circles around other drones. Because it mimics the unique and more flexible way bats fly, this 3-ounce prototype could do a better and safer job getting into disaster sites and scoping out construction zones than bulky drones with spinning rotors, said the three authors of a study released Wednesday in the journal Science Robotics. For example, it would have been ideal for going inside the damaged Fukushima nuclear plant in Japan, said study co-author Seth Hutchinson, an engineering professor at the University of Illinois. Bat Bot, a three-ounce flying robot can be more agile at getting into treacherous places than standard drones. The flying robot weighs just three ounces, and is equipped with nine joints. It measures about 8 inches from head to tail, and has a super-thin membrane that stretches to about a foot and a half.